Lisp Crash Course

This document is a very brief and short introduction course to Common Lisp, the programming language used for Kandria and many of its other systems. You are not expected to become a programmer or write code beyond some simple lines, so don't worry. Still, Lisp is different from other languages is a couple of ways, so try to go into this with a clear head and don't try to relate it to other languages you might already be familiar with.

If any questions come up, don't hesitate to contact Shinmera and ask for help. Seriously, do not hesitate.

Basic Syntax

Lisp syntax consists of a few basic constructs:

In addition to the syntax for these basic objects, Lisp notation includes a few shorthand constructs to make things easier to type. Most of these are indicated by the hash-sign prefixing them.

There's more syntax constructs in Lisp, but they are less frequently used and aren't too important.

Evaluation

Code isn't actually code unless it is evaluated or executed. The following terms are important to explain how Lisp code is executed, so we'll lay them out first:

Now, Lisp follows a fairly simple execution model, which we will discuss by looking at a couple of examples. If you've set up the development environment from the getting started guide, you can also try these out yourself by typing them into the prompt. When entering something into the REPL as it is called, the source code is parsed (or read), compiled, and evaluated. The result of evaluating your code is then printed back to you at the REPL.

"Hello World"

This is the most mundane and boring program you could write. It's the classic "Hello World" example, where a program just prints back Hello World at you. In Lisp with the REPL, the string is first read into a string object, which is then evaluated. Strings, like most objects, "evaluate to themselves". This means that the string is just returned as the result of the evaluation. Thus, you get the string back on the REPL.

You can do the same experiment with numbers and vectors, but not with lists or symbols. When evaluated as a form, lists and symbols carry special meaning.

(+ 1 2)

This program adds one and two together, and returns three.

Note how the list is used to express a function call. When this program is evaluated, Lisp sees a list and thus knows a function call is about to happen. It then looks at the first item in the list, which must be a symbol. In this case it's the symbol +, which denotes the addition function. It then looks at the remaining elements of the list and evaluates them in turn. Since both remaining list elements are numbers, they evaluate to themselves, thus we get 1 and 2 as the arguments to the function. The function + is then invoked with these arguments. The return value of this invocation is then returned as the result of the program.

The evaluation process is inherently recursive, as each argument to a function is evaluated. We can see this more clearly by looking at the following:

(/ (+ 2 3 4) 3)

This program computes the average of the numbers two, three, and four, which is three.

We make use of a new function here, /, which is the division operator. It divides the first argument by all remaining arguments. We also make use of +' ability to take an arbitrary number of arguments. Note the recursive nature of evaluation: first Lisp sees a list, looks at the / to find the function, then evaluates (+ 2 3 4). It sees another list, look sat the + to find the function, then evaluates 2, 3, and 4, calls + with the numbers as arguments, and uses the return value of 9 as the first argument to /. It then evaluates 3 and calls / with 9 and 3, finding the final return value of 3.

To summarise: in Lisp function calls are denoted by lists, where the first element in the list is the function name, and all remaining elements of the list are recursively evaluated to obtain the arguments to the function call.

We'll now quickly look at variables and bindings. As you might have noticed, symbols also play a special role in evaluation, but not only to denote function names, but also to denote variables. Let's look at the following example:

PI

This is another very simple program. When read, this is turned into the symbol PI. When a symbol is evaluated outside of the context of a function call, it is treated as a variable and whatever value is bound to the symbol is returned. In this case, the value of Pi is returned as a floating point number.

We can establish new variables with the special construct let. let allows us to introduce new variables for a limited part of the program (a scope). Unlike other functions, let is a special operator and thus can establish special semantic rules. Let's have a look at an example:

(let ((brother-age 25)
      (sister-age 27))
  (/ (+ brother-age sister-age) 2))

This program computes the average age of the two siblings, which is 26.

When Lisp sees a let call, it stops its normal evaluation rules and instead follows special rules. Namely, the second element of the list must be a list of variable bindings. Each element of this variable binding list must be another list that has two elements: a symbol naming the variable, and a value to bind to the new variable.

Thus in the above program we establish two new variable bindings, one called brother-age, and one called sister-age. These variable bindings are only active within the let call. After the variable bindings, let expects any number of other forms as its body. These forms now have access to the new variable bindings.

From here evaluation proceeds very similar to before, just instead of the numbers being arguments to + directly, we instead evaluate the symbols brother-age and sister-age, which will return 25 and 27 respectively, as that's what they were bound to.

let ultimately returns whatever the return value was of the last form in its body, in this case 26.

An example of an invalid program would be the following:

(let ((brother-age 25)
      (sister-age 27))
  (/ (+ brother-age sister-age) 2))
brother-age

In this case brother-age is referenced outside of the let that binds it. In the scope outside the let, Lisp does not know of any variable named brother-age and thus generates an error. When an unhandled error is signalled in Lisp, it will automatically open up a debugger. This allows you to look at the source of the error and try to fix it. Most times you'll simply want to abort the evaluation, so simply click on the lowest ABORT button, or press a within the debugger.

Let's keep the ante up and look at a more involved program:

(let ((ages (list 18 32 25 27 43)))
  (/ (reduce #'+ ages) (length ages)))

This program computes the average of the ages 18, 32, 25, 27, and 43, which happens to be 29.

Unlike previously though we dynamically compute the average. You can add or remove ages from the ages list and it'll keep computing the average no matter the length. To do this we use the function list, which simply constructs a list object from its arguments. We then bind this list to the ages variable with let. Then we sum up all of the elements of the list using reduce.

Here we see the first use of the #'FOO syntax. This syntax is the same as (FUNCTION FOO). function is another special operator like let, which when encountered returns the function object named by its argument. In this case it returns the function + so that we can use it as a value for reduce.

reduce is a function that takes another function and a list of elements as arguments. It then successively applies this other function to the elements of the list to "reduce" it to one value. In the above example it first evaluates (+ 18 32), yielding 50. It then evaluates (+ 50 25), yielding 75. It then evaluates (+ 75 27), yielding 102. Finally it evaluates (+ 102 43), yielding 145.

We then also evaluate (length ages). length is a function that returns how many elements a sequence has, in this case 5.

Thus we get to our final (/ 145 5), yielding the return value of 29.

The point of dynamic evaluation here might not be too poignant when written as a single program like this. However, we can factor out the body of our let into a function that can compute the average of any list it is given, instead. Defining new functions is done through another special operator called defun:

(defun average (ages)
  (/ (reduce #'+ ages) (length ages)))

defun first expects the name of the function to define, then a list that describes the arguments the function should expect, followed by a body of forms to evaluate when the function is invoked. Similar to let, the value returned by the last form in the body is used as the return value of the function.

Now that we have the averaging as a function, we can recreate our previous program like this:

(average (list 18 32 25 27 43))

At this point you may also have been confused by the difference between (list 1 2 3) and (1 2 3). If the latter is already a list anyway, what's the point of the list function? Can't we just write (1 2 3)?

The easiest answer is that normally Lisp expects the first item of a list to be a function name. 1 is not a function name, therefore (1 2 3) is not a valid Lisp form. We use the function list to construct a list dynamically and use the resulting list value instead.

However, there's another method to get a list as a value, which is through quotation. Quotation takes an expression you wrote in source code and turns it into a literal object, preventing it from being evaluated. Using quotation we can rewrite the previous program like this:

(average '(18 32 25 27 43))

The '(18 32 25 27 43) is short for (QUOTE (18 32 25 27 43)). quote is another special operator which, when evaluated, simply returns its argument as a literal value without evaluating it. Quotation is also often used to use symbols as names for things. By quoting the symbol it won't be treated as a variable, instead returning the symbol itself so that it can be used as a name. For example: 

(position 'name '(hello there my name is noodle))

This program returns the index of the symbol NAME within the list (HELLO THERE MY NAME IS NOODLE), which is 3.

Using symbols and lists we can also create association maps (dictionaries, or tables, whatever term you like best) by using lists composed out of a key and a value:

'((name "The Stranger")
  (age NIL)
  (location (100 2010))
  (health 90))

This is called an "association list", or "alist" for short. We can retrieve the value associated with the key like this:

(second (assoc 'name '((name "The Stranger")
                       (age NIL)
                       (location (100 2010))
                       (health 90))))

This program returns "The Stranger". The assoc function searches for a list that begins with the first arguments it is given in the elements of its second argument. In this case it returns (name "The stranger"). The second function then returns the second element of that list, "The Stranger".

Another common form of map is the "property list", or "plist". Instead of using a list for each entry in the map, keys and values are interleaved instead. The same map could thus be expressed as a plist like this:

(name "The Stranger"
 age NIL
 location (100 2010)
 health 90)

Rewriting the above program to use a plist, we get:

(getf '(name "The Stranger"
        age NIL
        location (100 2010)
        health 90)
      'name)

Often plists are written with keywords for the keys instead of regular symbols.

In general, the fact that lists, symbols, and other objects are so easy to access and create in Lisp source code makes it very tempting to use Lisp source, or in the very least its syntax, for data storage. Several of Kandria's data files make use of Lisp syntax to store data and code. Lisp being highly flexible and dynamic also makes it an ideal candidate as a scripting language itself, which is why whenever script-like things are needed in Kandria, Lisp code is used directly instead of creating or using a third-party language.

This concludes the most important aspects of evaluation in Lisp. To summarise again:

This completes the most important aspects of Lisp programming. Following are a few additional pieces of information that will be useful to know when dealing with code and data in Kandria.

Places

Another concept you'll encounter frequently in Lisp code is that of places. A place is something that can be read and written to in code. There's many different kinds of places that can be used, but the simplest is a variable. In order to modify any place, the setf special operator is used.

(let ((name "The Stranger"))
  (setf name "Fi")
  name)

This program returns "Fi", as the variable binding for name was modified with setf so that it was now bound to "Fi" rather than its original binding value of "The Stranger".

Another example of a place is the function nth that accesses the nth element of a list:

(let ((numbers (list 1 2 3 4 5)))
  (setf (nth 1 numbers) (nth 2 numbers))
  numbers)

A lot of things in Lisp can be modified as a place, and you can also add new places. For the purposes of this crash course, just remember that you can change stuff using setf.

Symbols

So far the symbols we've encountered have all been in "short form". Symbol syntax is actually quite a bit more involved, and each symbol also carries more properties than just its name. The only thing we'll look at here in relation to symbols is packages.

A symbol is a name that may be interned in a package. Symbols that are not interned in any package are called "uninterned symbols" and are written using the #:FOO syntax. Typically however, symbols are interned and are thus home to one or more packages. In order to reference a symbol from a specific package, you can use the extended syntax of package-name:symbol-name.

So far we had always used symbols without the package qualifier, in which case they are interned in the current package. The current package can be controlled and is often either explicitly set by the system, or declared at the beginning of a source file using in-package.

Whenever an unqualified symbol is read, Lisp checks whether a symbol of the requested name already exists in the current package. If it does, the existing symbol object is used. If not, a new symbol object of that name is created and interned into the current package.

For qualified symbols the behaviour is slightly different – if the symbol does not exist or is not exported, an error is signalled instead. In order for a symbol that is interned in a package to be accessible outside of it, it must thus be explicitly exported. This ensures that one cannot accidentally use symbols that were not meant to be used.

Symbols that are home to the package named KEYWORD can also be written by omitting the package name from the extended syntax, meaning :symbol-name. Keywords are quite frequently used to denote named arguments and other properties or attributes, as they are easy to access from any package. Any symbol interned in the keyword package is also automatically turned into a variable that holds the keyword itself as the value.

Useful Functions

This is a list of functions that are likely to be useful to you when writing small code snippets for the quest system.

More

This crash course only covers the bare essentials in very little depth. If you are interested enough to see a much more detailed and extensive coverage of Lisp's capabilities, please see the excellent Practical Common Lisp, available for free online.

Common Lisp also has an ANSI specification, which is the first place to look for a precise definition of what a function should do. It can be hard to read though.

When you're confused about something, the easiest way to get help is to just ask Shinmera.

Portacle

The development environment is built upon Portacle, an IDE for developing Common Lisp. Portacle has its own help file for help on essential terms and usage. For this we'll focus a bit on the Lisp related parts.

When starting up, the REPL should be open in one buffer and show the CL-USER> prompt. The CL-USER shows the current package (you can change the current package using ,i).

Typically you'll want to first load up Kandria. You can load "systems" (software packages) using the ql:quickload function.

(ql:quickload :kandria)

You can also do this any time the sources change even without restarting Portacle or Lisp. Should some kind of bug occur or the system get into a bad state, you can restart Lisp without restarting Portacle by typing ,restart.